COVID Detection using AI
The objective of this week is to experience how businesses use AI with tools & technologies involved to solve problems
The outcome of this week is not to understand the tiny details like codes and syntax, but to focus on how an end-to-end AI solution works, the methods, and the final output/results
Do not worry about the details of how something was created - you will learn the concepts demonstrated over the duration of this program
Think of this week as a preview of what you will be capable of doing at the end of this program
A renowned chain of hospitals is facing a problem of increased overload on resources (human as well as medical) due to a rise in the number of COVID cases. They want your help to build an AI solution that will help them predict the likelihood of a patient being infected by COVID by analyzing a chest X-ray scan image of the patient to identify the patients who are less likely to have COVID and prioritize critical cases.
First identified in December 2019 in the city of Wuhan in China, COVID is caused by a novel coronavirus known as SARS-CoV-2, after the initial outbreak, COVID rapidly spread within China and subsequently to other countries around the world. The World Health Organization (WHO) declared it a global pandemic in March 2020 due to its widespread transmission and impact on public health. Common symptoms of COVID include fever, cough, fatigue, shortness of breath, muscle or body aches, sore throat, loss of taste or smell, headache, and in severe cases, difficulty breathing or pneumonia. It's important to note that some individuals infected with SARS-CoV-2 may remain asymptomatic, meaning they show no noticeable symptoms but can still spread the virus to others. The severity of COVID can vary from mild to severe. Most individuals experience mild to moderate symptoms and recover without requiring hospitalization. Severe cases may require hospitalization, intensive care, and mechanical ventilation. Testing for COVID involves various methods, including molecular (PCR) and antigen tests, as well as chest X-ray scans.
Traditional testing methods (like the RT-PCR test), despite being the gold-standard, are time-consuming and resource-intensive. The RT-PCR test generally involves 3 stages (sample collection, extraction, PCR) with a 4-hour turnaround time (TAT) for the test results in general. This results in long wait times in detecting COVID and with an increase in the number of screenings, the wait time increases further.
With a rapid increase in the number of cases, hospitals face challenges in terms of staffing and logistics. The healthcare system is grappling with the need for swift and accurate COVID detection to curb the spread of the virus. Testing every case with mild symptoms becomes impractical and it becomes important to set up criteria for RT-PCR testing to reduce the turnaround time of test results and control the spread of the virus.
A renowned chain of hospitals have decided to leverage AI to deal with the crisis situation. They want to build an AI system to predict the chances of a patient being COVID positive by identifying patterns in chest X-ray scans.
The AI solution approach involves the scanning of chest X-rays of patients to detect patterns commonly found in COVID-infected people. Conducting a chest X-ray scan and obtaining the resultant scan image for a patient requires approximately 15 minutes. The chest X-rays can be passed to our AI solution and based on the likelihood of COVID.
This methodology can provide the following advantages to hospital:
The X-ray based detection can be used in conjunction with RT-PCR testing to increase the accuracy of diagnosis.
The data needed for buidling any AI solution is usually obtained from multiple sources.
In the current scenario, we have the following sources:
Medical Imaging Data: This includes infomration such as X-rays, CT scans, MRI scans, ultrasounds, and other types of medical images used for diagnosing and monitoring conditions.
Patient Health Records: This includes information such as patient medical history, medications, and test results.
Hospital Administrative Records: This includes data related to scheduling appointments, managing patient admissions and discharges, billing, insurance claims, and other administrative tasks.
Hospital Medical Resource Records: Includes information about medical supplies, equipment inventory, maintenance schedules, and usage. These records optimize resource allocation, enhance patient care, streamline operations, and ensure timely availability of critical medical assets while adhering to efficient inventory management practices.
Clinical Laboratory Test Data: Data generated from various medical tests, such as blood tests, urine tests, tissue samples, and genetic tests.
Patient Insurance Records: This includes data related to patients' health insurance coverage, claims, and reimbursement.
The data from different sources are collected and stored in an organized and secure manner in databases and PACS (Picture Archiving and Communication Systems) repositories. Once the data is stored, we can extract necessary data in multiple ways.
# installing the library for deploying AI models
!pip install gradio -q
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.4/50.4 kB 4.0 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.1/18.1 MB 105.5 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 318.7/318.7 kB 27.9 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 94.6/94.6 kB 7.9 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 76.4/76.4 kB 7.6 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 kB 7.3 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 141.9/141.9 kB 13.3 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.9/10.9 MB 112.2 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 5.7 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 5.3 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.5/71.5 kB 6.3 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.2/130.2 kB 10.1 MB/s eta 0:00:00
# this is necessary for training and building machine learning models
!pip install --upgrade tensorflow
Requirement already satisfied: tensorflow in /usr/local/lib/python3.10/dist-packages (2.17.0) Requirement already satisfied: absl-py>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.4.0) Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.6.3) Requirement already satisfied: flatbuffers>=24.3.25 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (24.3.25) Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (0.6.0) Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (0.2.0) Requirement already satisfied: h5py>=3.10.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (3.11.0) Requirement already satisfied: libclang>=13.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (18.1.1) Requirement already satisfied: ml-dtypes<0.5.0,>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (0.4.1) Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (3.3.0) Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from tensorflow) (24.1) Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (3.20.3) Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (2.32.3) Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from tensorflow) (71.0.4) Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.16.0) Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (2.4.0) Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (4.12.2) Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.16.0) Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.64.1) Requirement already satisfied: tensorboard<2.18,>=2.17 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (2.17.0) Requirement already satisfied: keras>=3.2.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (3.4.1) Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (0.37.1) Requirement already satisfied: numpy<2.0.0,>=1.23.5 in /usr/local/lib/python3.10/dist-packages (from tensorflow) (1.26.4) Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from astunparse>=1.6.0->tensorflow) (0.44.0) Requirement already satisfied: rich in /usr/local/lib/python3.10/dist-packages (from keras>=3.2.0->tensorflow) (13.8.1) Requirement already satisfied: namex in /usr/local/lib/python3.10/dist-packages (from keras>=3.2.0->tensorflow) (0.0.8) Requirement already satisfied: optree in /usr/local/lib/python3.10/dist-packages (from keras>=3.2.0->tensorflow) (0.12.1) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorflow) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorflow) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorflow) (2.2.3) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorflow) (2024.8.30) Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.18,>=2.17->tensorflow) (3.7) Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.18,>=2.17->tensorflow) (0.7.2) Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.18,>=2.17->tensorflow) (3.0.4) Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.10/dist-packages (from werkzeug>=1.0.1->tensorboard<2.18,>=2.17->tensorflow) (2.1.5) Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/dist-packages (from rich->keras>=3.2.0->tensorflow) (3.0.0) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/dist-packages (from rich->keras>=3.2.0->tensorflow) (2.18.0) Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py>=2.2.0->rich->keras>=3.2.0->tensorflow) (0.1.2)
# Importing libraries
from IPython import display # Enhanced interactive Python shell
import numpy as np # Numerical operations with arrays and matrices
import pandas as pd # Data manipulation and analysis library
import tensorflow as tf # Framework that provides tools for building machine learning models.
import os # Interact with file systems
import math # Mathematical functions and utilities
# Exploratory data analysis
import matplotlib.pyplot as plt # Data visualization library
import seaborn as sns # Statistical graphics
from tensorflow.keras.preprocessing.image import ImageDataGenerator # Image data generator with data augmentation
import cv2 # Image processing library
from PIL import ImageFilter # Image filters for image manipulation
# Model building
from tensorflow.keras.layers import Normalization, TextVectorization, StringLookup # Specific preprocessing layers
from sklearn.metrics import recall_score, confusion_matrix # Libraries for model evaluation
# Model deployment
import joblib # Save and load machine learning models
import gradio as gr # UI for model deployment
# To generate reproducible results
import random as python_random # Generate random data
np.random.seed(42) # Set seed for NumPy random generator
tf.random.set_seed(42) # Set seed for TensorFlow random generator
# Importing using Google Colab
# Connecting Google Drive to this Python notebook
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
# defining the dimensions of the images
height, width = 224, 224
# defining how many images to load in one go
batch_size=64
def generate_data(DIR):
# ImageDataGenerator is a function from tensorflow to perform rescaling, resizing, rotation, and other transformations.
datagen = ImageDataGenerator(rescale=1./255.)
# flow from directory is used to generate batches of image data from a directory structure, allowing for efficient loading and preprocessing of image data during model training or evaluation.
generator = datagen.flow_from_directory(
DIR,
batch_size=batch_size,
shuffle=True, # shuffles the order of images while generation
seed=42,
class_mode='sparse',
target_size=(height, width),
classes={'Normal': 0, 'Viral Pneumonia': 1,'Covid': 2}
)
return generator
# defining the file path where training and testing images are stored
TRAINING_DIR = '/content/drive/My Drive/Texas McCombs/Python Foundations/Case Studies/CaseStudy - COVID Detection/X-ray Data/train'
TESTING_DIR = '/content/drive/My Drive/Texas McCombs/Python Foundations/Case Studies/CaseStudy - COVID Detection/X-ray Data/test'
train_data = generate_data(TRAINING_DIR) # generate data for training
test_data = generate_data(TESTING_DIR) # generate data for testing
total_image = np.concatenate([train_data.labels,test_data.labels]) # used to access the labels associated with the images
Found 231 images belonging to 3 classes. Found 66 images belonging to 3 classes.
Exploratory Data Analysis (EDA) plays a very important role in an end-to-end AI solution. It enables
counts = {
'Normal': len(np.where(total_image == 0)[0]), # Calculating the count of Normal images
'Viral Pneumonia': len(np.where(total_image == 1)[0]), # Calculating the count of Viral Pneumonia images
'COVID': len(np.where(total_image == 2)[0]) # Calculating the count of COVID images
}
# Extract class labels and corresponding counts
class_labels = list(counts.keys())
class_counts = list(counts.values())
# Create the bar plot
plt.bar(class_labels, class_counts)
plt.xlabel('Classes')
plt.ylabel('Counts')
plt.title('Distribution of Image Types');
def image_plot(generator, images_per_class):
# Get the mapping of class names to indices
class_indices = generator.class_indices
# Extract the class names from the indices
class_names = list(class_indices.keys())
# Create a figure to display the images
plt.figure(figsize=(15, 10))
# Loop through each class index and its corresponding class name
for class_idx, class_name in enumerate(class_names):
# Get the indices of images belonging to the current class
class_indices = [i for i, value in enumerate(generator.classes) if value == class_idx]
# Select a subset of indices for the current class based on 'images_per_class'
selected_indices = class_indices[:images_per_class]
# Loop through the selected indices and display the images
for i, idx in enumerate(selected_indices):
# Create a subplot for each image
ax = plt.subplot(len(class_names), images_per_class, class_idx * images_per_class + i + 1)
# Load and display the image
img = plt.imread(generator.filepaths[idx])
plt.imshow(img)
plt.title(class_name)
plt.axis("off")
# Assuming 'train_data' is your image generator
images_per_class = 2
image_plot(train_data, images_per_class)
plt.show()
Distribution of Image Types
Types of Images
Identifying COVID vs non-COVID cases through manual observation of medical images poses a formidable challenge due to the subtle visual differences. The human eye's limitations can lead to extended diagnosis times and potential inaccuracies, especially when dealing with a considerable volume of patient data. Additionally, this process can be time-consuming and prone to subjectivity.
So, we need an AI model that can do the following:
The AI model is the 'heart' of our AI solution. The model serves as the core component that brings intelligence and functionality to an end-to-end AI solution. It leverages learned patterns and insights to generate predictions or perform tasks, enabling organizations to make data-driven decisions, automate processes, and unlock valuable insights from their data.
The model building step of an AI solution can be further broken down into the sub-steps shown below.
Training an AI model is important because it allows machines to learn and perform tasks without explicit programming. It enables the following:
# defining the model
tf.keras.backend.clear_session() # Clears the Keras session to remove any existing models or layers
input_shape = (height, width, 3) # Defines the input shape for the model (height, width, number of channels)
base_model = tf.keras.applications.vgg16.VGG16(
weights='imagenet', # Loads the pre-trained weights of the VGG16 model trained on ImageNet dataset
include_top=False, # Excludes the top (fully connected) layers of the VGG16 model
input_shape=input_shape
)
base_model.trainable = False # Freezes the weights of the VGG16 model to prevent further training
ai_model = tf.keras.Sequential() # Creates a sequential model
ai_model.add(base_model) # Adds the VGG16 base model to the sequential model
ai_model.add(tf.keras.layers.GlobalAveragePooling2D()) # Adds a global average pooling layer
ai_model.add(tf.keras.layers.Flatten()) # Flattens the input for the subsequent fully connected
ai_model.add(tf.keras.layers.Dense(3, activation='softmax')) # Adds the final fully connected layer with 3 units and softmax activation
ai_model.compile(loss='SparseCategoricalCrossentropy', # Specifies the loss function for trainin
optimizer=tf.keras.optimizers.Adam(0.001), # Sets the optimizer (Adam) and learning rate
metrics=['acc']) # Specifies the metrics to evaluate the model's performance
ai_model.summary()
Model Training
import random
seed_value = 42
random.seed(seed_value)
np.random.seed(seed_value)
tf.random.set_seed(seed_value)
checkpoint = tf.keras.callbacks.ModelCheckpoint('model/ai_model_best.saved', monitor='acc', verbose=1, mode='max',save_best_only=True) # defines file path to save the best model weights, define accuracy metric, verbose display updates of the model and saves best model
early = tf.keras.callbacks.EarlyStopping(monitor="val_loss", mode="min",restore_best_weights=True, patience=5) #stops the model training if there is no improvement in metrics
callbacks_list = [checkpoint,early]
history = ai_model.fit( # Fit the model on training data
train_data, # Training data generator
validation_data = test_data, # validation data generator
epochs=25, # no of epochs to train the model
shuffle=False,
verbose=True, # Prints updates during training.
callbacks=callbacks_list
)
Epoch 1/25 4/4 [==============================] - ETA: 0s - loss: 1.4845 - acc: 0.2597 Epoch 1: acc improved from -inf to 0.25974, saving model to model/ai_model_best.saved 4/4 [==============================] - 31s 4s/step - loss: 1.4845 - acc: 0.2597 - val_loss: 1.3230 - val_acc: 0.3030 Epoch 2/25 4/4 [==============================] - ETA: 0s - loss: 1.2610 - acc: 0.2597 Epoch 2: acc did not improve from 0.25974 4/4 [==============================] - 7s 2s/step - loss: 1.2610 - acc: 0.2597 - val_loss: 1.1859 - val_acc: 0.3030 Epoch 3/25 4/4 [==============================] - ETA: 0s - loss: 1.1253 - acc: 0.2771 Epoch 3: acc improved from 0.25974 to 0.27706, saving model to model/ai_model_best.saved 4/4 [==============================] - 7s 2s/step - loss: 1.1253 - acc: 0.2771 - val_loss: 1.1197 - val_acc: 0.2879 Epoch 4/25 4/4 [==============================] - ETA: 0s - loss: 1.0578 - acc: 0.4545 Epoch 4: acc improved from 0.27706 to 0.45455, saving model to model/ai_model_best.saved 4/4 [==============================] - 9s 2s/step - loss: 1.0578 - acc: 0.4545 - val_loss: 1.0973 - val_acc: 0.3939 Epoch 5/25 4/4 [==============================] - ETA: 0s - loss: 1.0306 - acc: 0.4805 Epoch 5: acc improved from 0.45455 to 0.48052, saving model to model/ai_model_best.saved 4/4 [==============================] - 7s 2s/step - loss: 1.0306 - acc: 0.4805 - val_loss: 1.0912 - val_acc: 0.3939 Epoch 6/25 4/4 [==============================] - ETA: 0s - loss: 1.0126 - acc: 0.4805 Epoch 6: acc did not improve from 0.48052 4/4 [==============================] - 7s 2s/step - loss: 1.0126 - acc: 0.4805 - val_loss: 1.0833 - val_acc: 0.3939 Epoch 7/25 4/4 [==============================] - ETA: 0s - loss: 0.9999 - acc: 0.4805 Epoch 7: acc did not improve from 0.48052 4/4 [==============================] - 6s 1s/step - loss: 0.9999 - acc: 0.4805 - val_loss: 1.0691 - val_acc: 0.3939 Epoch 8/25 4/4 [==============================] - ETA: 0s - loss: 0.9833 - acc: 0.4805 Epoch 8: acc did not improve from 0.48052 4/4 [==============================] - 5s 1s/step - loss: 0.9833 - acc: 0.4805 - val_loss: 1.0494 - val_acc: 0.3939 Epoch 9/25 4/4 [==============================] - ETA: 0s - loss: 0.9635 - acc: 0.4805 Epoch 9: acc did not improve from 0.48052 4/4 [==============================] - 6s 1s/step - loss: 0.9635 - acc: 0.4805 - val_loss: 1.0256 - val_acc: 0.3939 Epoch 10/25 4/4 [==============================] - ETA: 0s - loss: 0.9431 - acc: 0.4805 Epoch 10: acc did not improve from 0.48052 4/4 [==============================] - 9s 3s/step - loss: 0.9431 - acc: 0.4805 - val_loss: 1.0012 - val_acc: 0.3939 Epoch 11/25 4/4 [==============================] - ETA: 0s - loss: 0.9239 - acc: 0.4848 Epoch 11: acc improved from 0.48052 to 0.48485, saving model to model/ai_model_best.saved 4/4 [==============================] - 7s 2s/step - loss: 0.9239 - acc: 0.4848 - val_loss: 0.9797 - val_acc: 0.3939 Epoch 12/25 4/4 [==============================] - ETA: 0s - loss: 0.9061 - acc: 0.5065 Epoch 12: acc improved from 0.48485 to 0.50649, saving model to model/ai_model_best.saved 4/4 [==============================] - 7s 2s/step - loss: 0.9061 - acc: 0.5065 - val_loss: 0.9617 - val_acc: 0.3939 Epoch 13/25 4/4 [==============================] - ETA: 0s - loss: 0.8895 - acc: 0.5628 Epoch 13: acc improved from 0.50649 to 0.56277, saving model to model/ai_model_best.saved 4/4 [==============================] - 9s 2s/step - loss: 0.8895 - acc: 0.5628 - val_loss: 0.9452 - val_acc: 0.4394 Epoch 14/25 4/4 [==============================] - ETA: 0s - loss: 0.8749 - acc: 0.6277 Epoch 14: acc improved from 0.56277 to 0.62771, saving model to model/ai_model_best.saved 4/4 [==============================] - 7s 2s/step - loss: 0.8749 - acc: 0.6277 - val_loss: 0.9301 - val_acc: 0.4697 Epoch 15/25 4/4 [==============================] - ETA: 0s - loss: 0.8599 - acc: 0.6537 Epoch 15: acc improved from 0.62771 to 0.65368, saving model to model/ai_model_best.saved 4/4 [==============================] - 9s 2s/step - loss: 0.8599 - acc: 0.6537 - val_loss: 0.9181 - val_acc: 0.4697 Epoch 16/25 4/4 [==============================] - ETA: 0s - loss: 0.8452 - acc: 0.6537 Epoch 16: acc did not improve from 0.65368 4/4 [==============================] - 5s 1s/step - loss: 0.8452 - acc: 0.6537 - val_loss: 0.9049 - val_acc: 0.4848 Epoch 17/25 4/4 [==============================] - ETA: 0s - loss: 0.8301 - acc: 0.6623 Epoch 17: acc improved from 0.65368 to 0.66234, saving model to model/ai_model_best.saved 4/4 [==============================] - 8s 2s/step - loss: 0.8301 - acc: 0.6623 - val_loss: 0.8942 - val_acc: 0.5152 Epoch 18/25 4/4 [==============================] - ETA: 0s - loss: 0.8172 - acc: 0.6623 Epoch 18: acc did not improve from 0.66234 4/4 [==============================] - 6s 1s/step - loss: 0.8172 - acc: 0.6623 - val_loss: 0.8835 - val_acc: 0.5303 Epoch 19/25 4/4 [==============================] - ETA: 0s - loss: 0.8038 - acc: 0.6623 Epoch 19: acc did not improve from 0.66234 4/4 [==============================] - 6s 1s/step - loss: 0.8038 - acc: 0.6623 - val_loss: 0.8698 - val_acc: 0.6061 Epoch 20/25 4/4 [==============================] - ETA: 0s - loss: 0.7906 - acc: 0.7013 Epoch 20: acc improved from 0.66234 to 0.70130, saving model to model/ai_model_best.saved 4/4 [==============================] - 7s 2s/step - loss: 0.7906 - acc: 0.7013 - val_loss: 0.8569 - val_acc: 0.6515 Epoch 21/25 4/4 [==============================] - ETA: 0s - loss: 0.7779 - acc: 0.7273 Epoch 21: acc improved from 0.70130 to 0.72727, saving model to model/ai_model_best.saved 4/4 [==============================] - 7s 2s/step - loss: 0.7779 - acc: 0.7273 - val_loss: 0.8445 - val_acc: 0.6667 Epoch 22/25 4/4 [==============================] - ETA: 0s - loss: 0.7658 - acc: 0.7532 Epoch 22: acc improved from 0.72727 to 0.75325, saving model to model/ai_model_best.saved 4/4 [==============================] - 8s 2s/step - loss: 0.7658 - acc: 0.7532 - val_loss: 0.8325 - val_acc: 0.6818 Epoch 23/25 4/4 [==============================] - ETA: 0s - loss: 0.7541 - acc: 0.7879 Epoch 23: acc improved from 0.75325 to 0.78788, saving model to model/ai_model_best.saved 4/4 [==============================] - 7s 2s/step - loss: 0.7541 - acc: 0.7879 - val_loss: 0.8205 - val_acc: 0.7121 Epoch 24/25 4/4 [==============================] - ETA: 0s - loss: 0.7438 - acc: 0.8398 Epoch 24: acc improved from 0.78788 to 0.83983, saving model to model/ai_model_best.saved 4/4 [==============================] - 9s 2s/step - loss: 0.7438 - acc: 0.8398 - val_loss: 0.8085 - val_acc: 0.7727 Epoch 25/25 4/4 [==============================] - ETA: 0s - loss: 0.7324 - acc: 0.8442 Epoch 25: acc improved from 0.83983 to 0.84416, saving model to model/ai_model_best.saved 4/4 [==============================] - 7s 2s/step - loss: 0.7324 - acc: 0.8442 - val_loss: 0.7989 - val_acc: 0.7727
Model Evaluation
#model accuracy
ytrain = np.array([]) # Initialize an empty array for storing true labels of test data
xtrain = [] # Initialize an empty list for storing test data
for i in range(math.ceil(len(train_data.classes)/batch_size)): # Loop over test generator batches to extract test data and true labels
xtrain.append(train_data[i][0]) # Append test data to xtest list
ytrain= np.concatenate((ytrain,train_data[i][-1])) # Concatenate true labels to ytest array
xtrain = np.concatenate((xtrain),axis=0) # Concatenate test data along the batch axis
ypred_prob_train =ai_model.predict(xtrain) # Predict probabilities for the test data
ypred_train = np.argmax(ypred_prob_train,axis=1) # Predicted labels by selecting the class with the highest probability
8/8 [==============================] - 9s 405ms/step
model_train_score = recall_score(ytrain, ypred_train, average='macro')
print("Model Score on Train Data:", np.round(100*model_train_score, 2))
Model Score on Train Data: 81.88
# Set the size of the figure for the heatmap
plt.figure(figsize=(6, 6))
# Compute the confusion matrix based on true and predicted labels
hm = sns.heatmap(confusion_matrix(ytrain,ypred_train), annot=True, vmin=0, fmt='g', cmap='Blues', cbar=False,xticklabels=['Normal','Viral Penumonia','Covid'],yticklabels=['Normal','Viral Penumonia','Covid'])
hm.set(xlabel='Predicted labels') # Set the x-axis label for the heatmap
hm.set(ylabel='True label') # # Set the y-axis label for the heatmap
[Text(45.72222222222221, 0.5, 'True label')]
#model accuracy
ytest = np.array([]) # Initialize an empty array for storing true labels of test data
xtest = [] # Initialize an empty list for storing test data
for i in range(math.ceil(len(test_data.classes)/batch_size)): # Loop over test generator batches to extract test data and true labels
xtest.append(test_data[i][0]) # Append test data to xtest list
ytest= np.concatenate((ytest,test_data[i][-1])) # Concatenate true labels to ytest array
xtest = np.concatenate((xtest),axis=0) # Concatenate test data along the batch axis
ypred_prob_test =ai_model.predict(xtest) # Predict probabilities for the test data
ypred_test = np.argmax(ypred_prob_test,axis=1) # Predicted labels by selecting the class with the highest probability
3/3 [==============================] - 0s 109ms/step
model_test_score = recall_score(ytest, ypred_test, average='macro')
print("Model Score on test Data:", np.round(100*model_train_score, 2))
Model Score on test Data: 81.88
# Set the size of the figure for the heatmap
plt.figure(figsize=(6, 6))
# Compute the confusion matrix based on true and predicted labels
hm = sns.heatmap(confusion_matrix(ytest, ypred_test), annot=True, vmin=0, fmt='g', cmap='Blues', cbar=False,xticklabels=['Normal','Viral Penumonia','Covid'],yticklabels=['Normal','Viral Penumonia','Covid'])
hm.set(xlabel='Predicted labels') # Set the x-axis label for the heatmap
hm.set(ylabel='True label') # # Set the y-axis label for the heatmap
[Text(45.72222222222221, 0.5, 'True label')]
It is much more important to ensure that patients who actually have the infection are correctly identified based on their X-ray scan, even if it results in a few false COVID detections
Our AI model has a score of 84% on the train data and 75% on the test data respectively.
This score is pretty low considering our current scenario
This becomes a worry as the ultimate goal would be to make predictions for new patients, and we do not want a model that will fail to predict COVID cases then.
Model tuning is important for
In the current scenario, we are going to undertake two steps to tune the model
Since the number of images to train the data with is low, we will be 'synthetically creating' new data by applying various transformations to the existing images. These transformations create new variations of the data while maintaining the original label, resulting in a more diverse and robust training dataset. This helps in
Increased Generalization: By exposing it to different variations of the same data, the model generalizes better to new, unseen data. This helps prevent the scenario where the model becomes too specialized to the training data and performs poorly on new data.
Improved Robustness: By introducing variations in the training data, models become more robust to different lighting conditions, orientations, scales, and other factors that might be encountered in real-world scenarios.
# function to visualize the images
def image_plot(generator, image_numbers):
img_feature = generator[0][0][:image_numbers]
img_label = generator[0][1][:image_numbers]
plt.figure(figsize=(20, 15))
for i in range(image_numbers):
ax = plt.subplot(2, 3, i + 1) # Use 2 rows and 3 columns for the subplot grid
plt.imshow(img_feature[i])
plt.title("Normal" if img_label[i] == 0 else "Viral Pneumonia" if img_label[i] == 1 else "Covid")
plt.axis("off")
# adding image augmentation
def generate_data_augmented(DIR):
datagen = ImageDataGenerator( # Image data generator is a function from tensorflow to perform rescaling, resizing, rotation, and other transformations.
rescale=1./255., # Rescale pixel values to [0, 1]
zoom_range=0.2, # Randomly zooms images by a factor of 0.1
rotation_range=25, # Randomly rotates images by up to 20 degrees
width_shift_range=0.15, # Randomly shifts images horizontally by 10% of the total width
height_shift_range=0.15, # Randomly shifts images vertically by 10% of the total height
horizontal_flip = True # Randomly flips images horizontally
)
generator = datagen.flow_from_directory(
TRAINING_DIR, # Directory containing the training images
batch_size=batch_size, # Batch size for generating augmented data
seed=42, # Seed value for random number generation
class_mode='binary', # Type of class assignment ('binary' in this case)
target_size=(height, width), # Desired size (height, width) for the images
classes={'Normal': 0, 'Viral Pneumonia': 1,'Covid': 2} # Mapping of class names to numerical labels
)
return generator
aug_train_data = generate_data_augmented(TRAINING_DIR)
image_plot(aug_train_data,6)
Found 231 images belonging to 3 classes.
# defining the model architecture
tf.keras.backend.clear_session() # Clears the Keras session to remove any existing models or layers
input_shape = (height, width, 3) # Defines the input shape for the model (height, width, number of channels)
base_model = tf.keras.applications.vgg16.VGG16(
weights='imagenet', # Loads the pre-trained weights of the VGG16 model trained on ImageNet dataset
include_top=False, # Excludes the top (fully connected) layers of the VGG16 model
input_shape=input_shape
)
base_model.trainable = False # Freezes the weights of the VGG16 model to prevent further training
tuned_ai_model = tf.keras.Sequential() # Creates a sequential model
tuned_ai_model.add(base_model) # Adds the VGG16 base model to the sequential model
tuned_ai_model.add(tf.keras.layers.GlobalAveragePooling2D()) # Adds a global average pooling layer
tuned_ai_model.add(tf.keras.layers.Flatten()) # Flattens the input for the subsequent fully connected
tuned_ai_model.add(tf.keras.layers.Dense(256, activation='relu')) # Adds a fully connected layer with 256 units and ReLU activation
tuned_ai_model.add(tf.keras.layers.Dropout(0.5)) # Applies dropout regularization to prevent overfitting
tuned_ai_model.add(tf.keras.layers.Dense(256, activation='relu'))
tuned_ai_model.add(tf.keras.layers.Dropout(0.5))
tuned_ai_model.add(tf.keras.layers.Dense(3, activation='softmax')) # Adds the final fully connected layer with 3 units and softmax activation
tuned_ai_model.compile(loss='SparseCategoricalCrossentropy', # Specifies the loss function for trainin
optimizer=tf.keras.optimizers.Adam(0.001), # Sets the optimizer (Adam) and learning rate
metrics=['acc']) # Specifies the metrics to evaluate the model's performance
tuned_ai_model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Functional) (None, 7, 7, 512) 14714688
global_average_pooling2d ( (None, 512) 0
GlobalAveragePooling2D)
flatten (Flatten) (None, 512) 0
dense (Dense) (None, 256) 131328
dropout (Dropout) (None, 256) 0
dense_1 (Dense) (None, 256) 65792
dropout_1 (Dropout) (None, 256) 0
dense_2 (Dense) (None, 3) 771
=================================================================
Total params: 14912579 (56.89 MB)
Trainable params: 197891 (773.01 KB)
Non-trainable params: 14714688 (56.13 MB)
_________________________________________________________________
seed_value = 42
random.seed(seed_value)
np.random.seed(seed_value)
tf.random.set_seed(seed_value)
# defines file path to save the best model weights, define accuracy metric, verbose display updates of the model and saves best model
checkpoint = tf.keras.callbacks.ModelCheckpoint('model/tuned_ai_model_best.saved', monitor='acc', verbose=1, mode='max',save_best_only=True)
# stops the model training if there is no improvement in metrics
early = tf.keras.callbacks.EarlyStopping(monitor="val_loss", mode="min",restore_best_weights=True, patience=10)
callbacks_list = [checkpoint,early]
# training the model
history = tuned_ai_model.fit( # fit the model on training data
aug_train_data,
validation_data = test_data,
epochs=50, # no of epochs to train the model
shuffle=False,
verbose=True, # Prints updates during training.
callbacks=callbacks_list
)
Epoch 1/50 4/4 [==============================] - ETA: 0s - loss: 1.2529 - acc: 0.3896 Epoch 1: acc improved from -inf to 0.38961, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 14s 3s/step - loss: 1.2529 - acc: 0.3896 - val_loss: 1.1280 - val_acc: 0.3939 Epoch 2/50 4/4 [==============================] - ETA: 0s - loss: 1.1196 - acc: 0.4416 Epoch 2: acc improved from 0.38961 to 0.44156, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 1.1196 - acc: 0.4416 - val_loss: 0.9905 - val_acc: 0.5000 Epoch 3/50 4/4 [==============================] - ETA: 0s - loss: 1.1166 - acc: 0.4242 Epoch 3: acc did not improve from 0.44156 4/4 [==============================] - 8s 2s/step - loss: 1.1166 - acc: 0.4242 - val_loss: 0.9414 - val_acc: 0.3939 Epoch 4/50 4/4 [==============================] - ETA: 0s - loss: 0.9544 - acc: 0.5455 Epoch 4: acc improved from 0.44156 to 0.54545, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.9544 - acc: 0.5455 - val_loss: 0.8936 - val_acc: 0.5606 Epoch 5/50 4/4 [==============================] - ETA: 0s - loss: 0.8810 - acc: 0.5714 Epoch 5: acc improved from 0.54545 to 0.57143, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.8810 - acc: 0.5714 - val_loss: 0.8254 - val_acc: 0.7879 Epoch 6/50 4/4 [==============================] - ETA: 0s - loss: 0.8534 - acc: 0.6017 Epoch 6: acc improved from 0.57143 to 0.60173, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 10s 3s/step - loss: 0.8534 - acc: 0.6017 - val_loss: 0.7559 - val_acc: 0.7727 Epoch 7/50 4/4 [==============================] - ETA: 0s - loss: 0.8136 - acc: 0.6061 Epoch 7: acc improved from 0.60173 to 0.60606, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.8136 - acc: 0.6061 - val_loss: 0.7016 - val_acc: 0.7121 Epoch 8/50 4/4 [==============================] - ETA: 0s - loss: 0.7333 - acc: 0.7056 Epoch 8: acc improved from 0.60606 to 0.70563, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 12s 3s/step - loss: 0.7333 - acc: 0.7056 - val_loss: 0.6376 - val_acc: 0.7424 Epoch 9/50 4/4 [==============================] - ETA: 0s - loss: 0.7231 - acc: 0.6926 Epoch 9: acc did not improve from 0.70563 4/4 [==============================] - 8s 2s/step - loss: 0.7231 - acc: 0.6926 - val_loss: 0.5969 - val_acc: 0.8030 Epoch 10/50 4/4 [==============================] - ETA: 0s - loss: 0.6417 - acc: 0.7273 Epoch 10: acc improved from 0.70563 to 0.72727, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 10s 3s/step - loss: 0.6417 - acc: 0.7273 - val_loss: 0.5676 - val_acc: 0.8788 Epoch 11/50 4/4 [==============================] - ETA: 0s - loss: 0.6532 - acc: 0.7143 Epoch 11: acc did not improve from 0.72727 4/4 [==============================] - 9s 2s/step - loss: 0.6532 - acc: 0.7143 - val_loss: 0.5287 - val_acc: 0.8030 Epoch 12/50 4/4 [==============================] - ETA: 0s - loss: 0.6107 - acc: 0.7532 Epoch 12: acc improved from 0.72727 to 0.75325, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.6107 - acc: 0.7532 - val_loss: 0.5269 - val_acc: 0.7727 Epoch 13/50 4/4 [==============================] - ETA: 0s - loss: 0.5414 - acc: 0.7403 Epoch 13: acc did not improve from 0.75325 4/4 [==============================] - 8s 2s/step - loss: 0.5414 - acc: 0.7403 - val_loss: 0.4948 - val_acc: 0.7727 Epoch 14/50 4/4 [==============================] - ETA: 0s - loss: 0.6182 - acc: 0.7489 Epoch 14: acc did not improve from 0.75325 4/4 [==============================] - 9s 2s/step - loss: 0.6182 - acc: 0.7489 - val_loss: 0.4592 - val_acc: 0.8788 Epoch 15/50 4/4 [==============================] - ETA: 0s - loss: 0.5038 - acc: 0.8225 Epoch 15: acc improved from 0.75325 to 0.82251, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 10s 3s/step - loss: 0.5038 - acc: 0.8225 - val_loss: 0.4482 - val_acc: 0.8333 Epoch 16/50 4/4 [==============================] - ETA: 0s - loss: 0.5679 - acc: 0.7749 Epoch 16: acc did not improve from 0.82251 4/4 [==============================] - 9s 2s/step - loss: 0.5679 - acc: 0.7749 - val_loss: 0.4419 - val_acc: 0.7879 Epoch 17/50 4/4 [==============================] - ETA: 0s - loss: 0.4788 - acc: 0.8268 Epoch 17: acc improved from 0.82251 to 0.82684, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.4788 - acc: 0.8268 - val_loss: 0.4309 - val_acc: 0.8182 Epoch 18/50 4/4 [==============================] - ETA: 0s - loss: 0.4669 - acc: 0.8095 Epoch 18: acc did not improve from 0.82684 4/4 [==============================] - 8s 2s/step - loss: 0.4669 - acc: 0.8095 - val_loss: 0.4124 - val_acc: 0.8485 Epoch 19/50 4/4 [==============================] - ETA: 0s - loss: 0.4539 - acc: 0.8355 Epoch 19: acc improved from 0.82684 to 0.83550, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.4539 - acc: 0.8355 - val_loss: 0.3955 - val_acc: 0.8485 Epoch 20/50 4/4 [==============================] - ETA: 0s - loss: 0.4495 - acc: 0.8442 Epoch 20: acc improved from 0.83550 to 0.84416, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.4495 - acc: 0.8442 - val_loss: 0.3390 - val_acc: 0.9091 Epoch 21/50 4/4 [==============================] - ETA: 0s - loss: 0.4438 - acc: 0.7965 Epoch 21: acc did not improve from 0.84416 4/4 [==============================] - 10s 2s/step - loss: 0.4438 - acc: 0.7965 - val_loss: 0.4142 - val_acc: 0.8182 Epoch 22/50 4/4 [==============================] - ETA: 0s - loss: 0.4630 - acc: 0.8268 Epoch 22: acc did not improve from 0.84416 4/4 [==============================] - 9s 2s/step - loss: 0.4630 - acc: 0.8268 - val_loss: 0.3786 - val_acc: 0.8636 Epoch 23/50 4/4 [==============================] - ETA: 0s - loss: 0.4295 - acc: 0.8398 Epoch 23: acc did not improve from 0.84416 4/4 [==============================] - 8s 2s/step - loss: 0.4295 - acc: 0.8398 - val_loss: 0.3200 - val_acc: 0.9242 Epoch 24/50 4/4 [==============================] - ETA: 0s - loss: 0.4406 - acc: 0.8312 Epoch 24: acc did not improve from 0.84416 4/4 [==============================] - 9s 2s/step - loss: 0.4406 - acc: 0.8312 - val_loss: 0.3242 - val_acc: 0.8788 Epoch 25/50 4/4 [==============================] - ETA: 0s - loss: 0.4235 - acc: 0.8442 Epoch 25: acc did not improve from 0.84416 4/4 [==============================] - 8s 2s/step - loss: 0.4235 - acc: 0.8442 - val_loss: 0.3058 - val_acc: 0.9091 Epoch 26/50 4/4 [==============================] - ETA: 0s - loss: 0.3849 - acc: 0.8485 Epoch 26: acc improved from 0.84416 to 0.84848, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 12s 3s/step - loss: 0.3849 - acc: 0.8485 - val_loss: 0.3554 - val_acc: 0.8485 Epoch 27/50 4/4 [==============================] - ETA: 0s - loss: 0.3862 - acc: 0.8658 Epoch 27: acc improved from 0.84848 to 0.86580, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 10s 3s/step - loss: 0.3862 - acc: 0.8658 - val_loss: 0.3457 - val_acc: 0.8636 Epoch 28/50 4/4 [==============================] - ETA: 0s - loss: 0.4212 - acc: 0.8268 Epoch 28: acc did not improve from 0.86580 4/4 [==============================] - 8s 2s/step - loss: 0.4212 - acc: 0.8268 - val_loss: 0.3169 - val_acc: 0.8636 Epoch 29/50 4/4 [==============================] - ETA: 0s - loss: 0.3753 - acc: 0.8615 Epoch 29: acc did not improve from 0.86580 4/4 [==============================] - 10s 2s/step - loss: 0.3753 - acc: 0.8615 - val_loss: 0.2831 - val_acc: 0.9242 Epoch 30/50 4/4 [==============================] - ETA: 0s - loss: 0.3731 - acc: 0.8615 Epoch 30: acc did not improve from 0.86580 4/4 [==============================] - 8s 2s/step - loss: 0.3731 - acc: 0.8615 - val_loss: 0.2829 - val_acc: 0.9242 Epoch 31/50 4/4 [==============================] - ETA: 0s - loss: 0.3541 - acc: 0.8571 Epoch 31: acc did not improve from 0.86580 4/4 [==============================] - 9s 2s/step - loss: 0.3541 - acc: 0.8571 - val_loss: 0.3898 - val_acc: 0.8485 Epoch 32/50 4/4 [==============================] - ETA: 0s - loss: 0.3886 - acc: 0.8701 Epoch 32: acc improved from 0.86580 to 0.87013, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.3886 - acc: 0.8701 - val_loss: 0.2987 - val_acc: 0.8636 Epoch 33/50 4/4 [==============================] - ETA: 0s - loss: 0.3290 - acc: 0.8615 Epoch 33: acc did not improve from 0.87013 4/4 [==============================] - 8s 2s/step - loss: 0.3290 - acc: 0.8615 - val_loss: 0.2649 - val_acc: 0.9242 Epoch 34/50 4/4 [==============================] - ETA: 0s - loss: 0.3790 - acc: 0.8528 Epoch 34: acc did not improve from 0.87013 4/4 [==============================] - 7s 2s/step - loss: 0.3790 - acc: 0.8528 - val_loss: 0.2823 - val_acc: 0.9091 Epoch 35/50 4/4 [==============================] - ETA: 0s - loss: 0.3554 - acc: 0.8701 Epoch 35: acc did not improve from 0.87013 4/4 [==============================] - 9s 2s/step - loss: 0.3554 - acc: 0.8701 - val_loss: 0.2919 - val_acc: 0.8636 Epoch 36/50 4/4 [==============================] - ETA: 0s - loss: 0.3349 - acc: 0.8701 Epoch 36: acc did not improve from 0.87013 4/4 [==============================] - 7s 2s/step - loss: 0.3349 - acc: 0.8701 - val_loss: 0.2772 - val_acc: 0.8788 Epoch 37/50 4/4 [==============================] - ETA: 0s - loss: 0.3280 - acc: 0.8745 Epoch 37: acc improved from 0.87013 to 0.87446, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 12s 3s/step - loss: 0.3280 - acc: 0.8745 - val_loss: 0.2984 - val_acc: 0.8636 Epoch 38/50 4/4 [==============================] - ETA: 0s - loss: 0.3329 - acc: 0.8571 Epoch 38: acc did not improve from 0.87446 4/4 [==============================] - 7s 2s/step - loss: 0.3329 - acc: 0.8571 - val_loss: 0.2996 - val_acc: 0.8636 Epoch 39/50 4/4 [==============================] - ETA: 0s - loss: 0.3068 - acc: 0.8874 Epoch 39: acc improved from 0.87446 to 0.88745, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.3068 - acc: 0.8874 - val_loss: 0.2438 - val_acc: 0.9394 Epoch 40/50 4/4 [==============================] - ETA: 0s - loss: 0.2756 - acc: 0.9091 Epoch 40: acc improved from 0.88745 to 0.90909, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.2756 - acc: 0.9091 - val_loss: 0.2760 - val_acc: 0.8788 Epoch 41/50 4/4 [==============================] - ETA: 0s - loss: 0.2485 - acc: 0.9134 Epoch 41: acc improved from 0.90909 to 0.91342, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.2485 - acc: 0.9134 - val_loss: 0.2650 - val_acc: 0.8939 Epoch 42/50 4/4 [==============================] - ETA: 0s - loss: 0.2913 - acc: 0.8874 Epoch 42: acc did not improve from 0.91342 4/4 [==============================] - 9s 2s/step - loss: 0.2913 - acc: 0.8874 - val_loss: 0.2480 - val_acc: 0.9091 Epoch 43/50 4/4 [==============================] - ETA: 0s - loss: 0.2733 - acc: 0.9048 Epoch 43: acc did not improve from 0.91342 4/4 [==============================] - 8s 2s/step - loss: 0.2733 - acc: 0.9048 - val_loss: 0.2343 - val_acc: 0.9242 Epoch 44/50 4/4 [==============================] - ETA: 0s - loss: 0.2729 - acc: 0.8961 Epoch 44: acc did not improve from 0.91342 4/4 [==============================] - 7s 2s/step - loss: 0.2729 - acc: 0.8961 - val_loss: 0.2316 - val_acc: 0.9242 Epoch 45/50 4/4 [==============================] - ETA: 0s - loss: 0.2519 - acc: 0.9004 Epoch 45: acc did not improve from 0.91342 4/4 [==============================] - 10s 2s/step - loss: 0.2519 - acc: 0.9004 - val_loss: 0.2498 - val_acc: 0.8788 Epoch 46/50 4/4 [==============================] - ETA: 0s - loss: 0.2413 - acc: 0.9221 Epoch 46: acc improved from 0.91342 to 0.92208, saving model to model/tuned_ai_model_best.saved 4/4 [==============================] - 11s 3s/step - loss: 0.2413 - acc: 0.9221 - val_loss: 0.3430 - val_acc: 0.8485 Epoch 47/50 4/4 [==============================] - ETA: 0s - loss: 0.2682 - acc: 0.9004 Epoch 47: acc did not improve from 0.92208 4/4 [==============================] - 8s 2s/step - loss: 0.2682 - acc: 0.9004 - val_loss: 0.2117 - val_acc: 0.9242 Epoch 48/50 4/4 [==============================] - ETA: 0s - loss: 0.2707 - acc: 0.8961 Epoch 48: acc did not improve from 0.92208 4/4 [==============================] - 9s 2s/step - loss: 0.2707 - acc: 0.8961 - val_loss: 0.2072 - val_acc: 0.9242 Epoch 49/50 4/4 [==============================] - ETA: 0s - loss: 0.2772 - acc: 0.9221 Epoch 49: acc did not improve from 0.92208 4/4 [==============================] - 8s 2s/step - loss: 0.2772 - acc: 0.9221 - val_loss: 0.3180 - val_acc: 0.8636 Epoch 50/50 4/4 [==============================] - ETA: 0s - loss: 0.2423 - acc: 0.9177 Epoch 50: acc did not improve from 0.92208 4/4 [==============================] - 9s 2s/step - loss: 0.2423 - acc: 0.9177 - val_loss: 0.2397 - val_acc: 0.9091
Model Evaluation
ytrain = np.array([]) # Initialize an empty array for storing true labels of test data
xtrain = [] # Initialize an empty list for storing test data
for i in range(math.ceil(len(aug_train_data.classes)/batch_size)): # Loop over test generator batches to extract test data and true labels
xtrain.append(aug_train_data[i][0]) # Append test data to xtest list
ytrain= np.concatenate((ytrain,aug_train_data[i][-1])) # Concatenate true labels to ytest array
xtrain = np.concatenate((xtrain),axis=0) # Concatenate test data along the batch axis
ypred_prob_train_tune = tuned_ai_model.predict(xtrain) # Predict probabilities for the test data
ypred_train_tune = np.argmax(ypred_prob_train_tune,axis=1) # Predicted labels by selecting the class with the highest probability
8/8 [==============================] - 1s 119ms/step
model_train_score = recall_score(ytrain, ypred_train_tune,average='macro')
print("Model Score on Train Data:", np.round(100*model_train_score, 2))
Model Score on Train Data: 91.07
# Set the size of the figure for the heatmap
plt.figure(figsize=(6, 6))
# Compute the confusion matrix based on true and predicted labels
hm = sns.heatmap(confusion_matrix(ytrain,ypred_train_tune), annot=True, vmin=0, fmt='g', cmap='Blues', cbar=False,xticklabels=['Normal','Viral Penumonia','Covid'],yticklabels=['Normal','Viral Penumonia','Covid'])
hm.set(xlabel='Predicted labels') # Set the x-axis label for the heatmap
hm.set(ylabel='True label') # # Set the y-axis label for the heatmap
[Text(45.72222222222221, 0.5, 'True label')]
ytest = np.array([]) # Initialize an empty array for storing true labels of test data
xtest = [] # Initialize an empty list for storing test data
for i in range(math.ceil(len(test_data.classes)/batch_size)): # Loop over test generator batches to extract test data and true labels
xtest.append(test_data[i][0]) # Append test data to xtest list
ytest= np.concatenate((ytest,test_data[i][-1])) # Concatenate true labels to ytest array
xtest = np.concatenate((xtest),axis=0) # Concatenate test data along the batch axis
ypred_prob_test_tune =tuned_ai_model.predict(xtest) # Predict probabilities for the test data
ypred_test_tune = np.argmax(ypred_prob_test_tune,axis=1) # Predicted labels by selecting the class with the highest probability
3/3 [==============================] - 0s 133ms/step
model_test_score = recall_score(ytest, ypred_test_tune,average='macro')
print("Model Score on Test Data:", np.round(100*model_test_score, 2))
Model Score on Test Data: 90.0
# Set the size of the figure for the heatmap
plt.figure(figsize=(6, 6))
# Compute the confusion matrix based on true and predicted labels
hm = sns.heatmap(confusion_matrix(ytest, ypred_test_tune), annot=True, vmin=0, fmt='g', cmap='Blues', cbar=False,xticklabels=['Normal','Viral Penumonia','Covid'],yticklabels=['Normal','Viral Penumonia','Covid'])
hm.set(xlabel='Predicted labels') # Set the x-axis label for the heatmap
hm.set(ylabel='True label') # # Set the y-axis label for the heatmap
[Text(45.72222222222221, 0.5, 'True label')]
Model testing is important for:
ytest = np.array([])
xtest = []
for i in range(math.ceil(len(test_data.classes)/batch_size)):
xtest.append(test_data[i][0])
ytest= np.concatenate((ytest,test_data[i][-1]))
xtest = np.concatenate((xtest),axis=0)
ypred_prob = tuned_ai_model.predict(xtest)
ypred = np.argmax(ypred_prob,axis=1)
model_test_score = recall_score(ytest, ypred_test_tune,average='macro')
print("Model Score on Test Data:", np.round(100*model_test_score, 2), '\n\n')
plt.figure(figsize=(6, 6))
hm = sns.heatmap(confusion_matrix(ytest,ypred), annot=True, vmin=0, fmt='g', cmap='Blues', cbar=False,
xticklabels=['Normal','Viral Penumonia','Covid'],yticklabels=['Normal','Viral Penumonia','Covid'])
hm.set(xlabel='Predicted_labels')
hm.set(ylabel='True_labels');
3/3 [==============================] - 0s 108ms/step Model Score on Test Data: 90.0
Model deployment helps with the following:
# path to the save the trained model
saved_model_path = "/content/drive/covid_detector.joblib"
# saving the final model
joblib.dump(tuned_ai_model, saved_model_path)
['/content/drive/MyDrive/Colab Notebooks/AI Application Case Study - Image Data/covid_detector.joblib']
# path to the save the trained model
saved_model_path = "/content/drive/covid_detector.joblib"
# loading the saved model
covid_detector = joblib.load(saved_model_path)
# creating the deployment input interface
image = gr.Image()
# creating the deployment output interface
label = gr.Label(num_top_classes=3)
# defining the dimensions of the input image
height = 224
width = 224
dimensions = (width, height)
# defining the class names for predictions
class_names = {0: 'Normal', 1: 'Viral Pneumonia', 2: 'Covid'}
# define a function that will take the necessary inputs and make predictions
def predict_covid(image):
# resizing the input image
image = cv2.resize(image, dimensions, interpolation=cv2.INTER_LINEAR)
image = image / 255.0
# reshaping the image to match the model's input shape
image = image.reshape((-1, 224, 224, 3))
# making predictions using the loaded model
prediction = covid_detector.predict(image).flatten()
# formatting the results to return final results as class names
return {class_names[i]: float(prediction[i]) for i in range(3)}
# defining the structure of the deployment interface and how the components will interact
demo = gr.Interface(
fn=predict_covid,
inputs = image,
outputs = label,
title="COVID Detection",
description= "This interface will predict whether a given patient is normal, has viral pneumonia, or has COVID based on chest X-ray scan provided.",
allow_flagging="never"
)
# deploying the model
demo.launch(inline=False, share=True, debug=True)
# shutting down the deployed model
demo.close()
There are generally two main modes of making predictions with a deployed AI model:
Batch Prediction: In batch prediction mode, predictions are made on a batch of input data all at once. This mode is suitable when you have a large set of data that needs predictions in a batch process, such as running predictions on historical data or performing bulk predictions on a scheduled basis.
Real-time (or Interactive) Prediction: In real-time or interactive prediction mode, predictions are made on individual data points in real-time as they arrive. This mode is suitable when you need immediate or on-demand predictions for new and incoming data.
The choice of prediction mode depends on the specific requirements and use case of the deployed AI model. Batch prediction is preferable when efficiency in processing large volumes of data is important, while real-time prediction is suitable for scenarios that require immediate or interactive responses to new data.
Metrics and dashboarding are the tools that businesses use to track their performance. Some of the benefits of using metrics and dashboarding:
Note: The above chart is indicative in nature.
We have built an AI model, tested it, deployed it, and used the model's outputs to visualize the important business metrics via dashboards. Now the final step is to use the AI model for decision-making and determine the impact of implementing the AI solution.
The trends of model performance useful for the Data Team. They can use it to
Healthcare personnel (doctors, radiologists, etc.) can use the AI solution to take a call on cases where the model's predictions are on the edge.
Let's assess the impact of our AI solution.
Note: The above numbers are indicative in nature.
**Happy Learning!**